I echo Michael Plantâs sentiments. Iâm glad youâre quantifying the benefits of this potential intervention.
I started looking through the CEA and thought it seemed optimistic in various ways, but then I realized I could just look at the end number and see if, even without adjustments, it beat GiveWell recommended charities. Unfortunately it doesnât. You said that GiveWellâs charities are in the range of hundreds of dollars per DALY and that didnât gel with my memory. I looked it up and AMF is around $1,965 per life saved, equivalent to 36 DALYs, so 1,965â36 = ~$54/âDALY. SCI was $1,080 per life saved, so $1,080/â36 = $30/âDALY. GiveDirectly is $11,663/â36=~$323/âDALY, but the reason they recommend GiveDirectly is in part because it has a lower barrier to prove itself in evidence because it is a very âdirectâ intervention. (source: https://ââdocs.google.com/ââspreadsheets/ââd/ââ13b_qt-G_TQtoYNznNak3_5dzvzgCSUPJnk3l5dMisJo/ââedit#gid=1034883018) These numbers hold up roughly when you look at estimates from 2012 when they were still using DALYs.
Nevertheless, if you are considering where to donate, your best guess estimate is less cost-effective than GiveWell interventions. This is before it goes through the rigors of a GiveWell CEA as well, which would definitely have less optimistic numbers, especially given the low evidence base.
Iâd like to end on a note that I think that posting new cause areas to the EA movement is scary because itâs a critical minded bunch, so hats off to you for having the courage to do so. Keep trying; I commend you for it. Unfortunately, even if you defend it as well as it can be defended, it might not win compared to the existing top charities. However, if nobody does this work, no new causes will be âdiscoveredâ, and so even if it doesnât win, this sort of work is very likely to be net good in expectation.
The reason why interventions like AR or x-risk are accepted by the EA movement (although not by all EAs) is that from a CEA perspective they do better than GiveWell top charities. The reason a lot of people still donât accept them as interventions though is because people discount based on evidence base differently, with some people taking non-evidence based CEAs more seriously than others. If drug policy does worse from a CEA perspective than GiveWell, AR and x-risk, and is worse from an evidence perspective than GiveWell charities, where is its advantage?
You could make a case that itâs better from a metric perspective (ie preventing unhappiness through depression rather than DALYs which has issues with it, like over-valuing preventing death according to a lot of value systems), but deworming improves lives; it doesnât prevent death. Same with GiveDirectly.
For giving detailed feedback on the CEA, I unfortunately just donât have the energy to do the full thing, but if the final number still isnât enough to make me switch from GiveWell charities, it doesnât make sense to look more into the details. However, one thing that jumped out to me that others mentioned was the chance of the ballot coming through. I think looking up the historical rate of ballot initiatives being passed would be a good thing to look into.
On your original comparison to GW charities, I wouldnât just take GWâs analysis as the canonic truth on the matter. Their CEA is pretty complicated, but ultimately they value charities based on how well they either 1. save lives or 2. increase consumption.
What you think about the value of saving lives is a philosophical question. Iâve written about this elsewhere so wonât repeat myself.
What you think about the value of increase consumption (SCI and GD) is probably an empirical question. If you value happiness then increasing consumption is a really bad way to increase aggregate happiness because of adaption and comparison effects (I discuss this in my EAG talk).
When I think about GW charities, Iâm am highly sceptical they do much good at all. I know, highly controversial....
I say this because it opens the space to look at other things, like mental health and pain, both of which drug policy reform help.
However, even if you take GWâs calcs at face value (Iâm not sure exactly what that is) I think it would still be possible to build an EV calc for drug policy reform that rivals them. In this post I suggest a campaign for rescheduling psychedelics could spend ÂŁ250 billion and be competitive with AMF. Milanâs calcs are really helpful because itâs important to start filling in the details of this analysis. In many ways, Milanâs is more complete than mine, which is quite simple.
Fair point, that deworming and cash transfers increase consumption instead of directly increase well being, or at least thatâs what GiveWellâs main analysis rests on. I do recall that the GD study actually did look at SWB and on page 4 (bit.ly/ââ2B97A1Y) it says that it increased a bunch of different happiness metrics as well (depression, stress, happiness and life satisfaction). However, if you only looked at that effect, GiveDirectly may not be that cost-effective. I havenât investigated it that much from that angle.
In terms of preventing infant mortality, it seems unlikely that losing a child wouldnât cause immense suffering to the parents, especially the mother. People often think that this wouldnât happen because people just âget used toâ babies dying, but the odds that a child will die is actually quite low nowadays, even in the developing world. In India, where I have the most experience, itâs measured in deaths per 1,000 live births, not 100, because itâs thatâs rare. Additionally, because I donât think death is nearly as bad as DALYs would have it, I looked a lot into parental mourning before choosing SMS reminders. I donât have anything formal I wrote up I can point to (though I might at some point), but my research found that most parents, after the loss of a child, are depressed for around a year, with some tail ends of people who never appear to recover.
If itâs the metrics issue thatâs leading to drug policy reform, I would recommend looking into preventing iron deficiency (through supplements or fortification) as an alternative. Itâs more evidence based and iron deficiency causes massive unhappiness. Anecdotally Iâve had friends who transformed from sad grumpy monsters into happy productive members of society after realizing they were deficient. Additionally thereâs evidence it increases income, increases IQ if taken during pregnancy, and decreases mortality in certain circumstances, so itâs pretty robust no matter the metrics you care about.
Lastly, Iâll admit that I havenât read all of your posts /â critiques of AMFâs effectiveness, so Iâll have to go and do that :)
The basic thrust is that psychedelic experiences can function as a catalytic engine for social change, both by improving the motivations of highly capable but insufficiently reflective people, and by improving the capabilities of well-intentioned people who struggle with internal blockers.
If thatâs what you think does the real work of drug liberalisation, you should probably state that and build a cost-effectiveness model on that basis, rather than try to justify drug reform on other terms but with that as the true motivation. I, for one, am pretty sceptical, because I canât imagine loads more people would, counterfactually, start taking drugs recreationally and that, for those that do, this will have much impact on their cognitive powers.
Hmm. Yes, I agree cognitive shifts could be pretty powerful from psychedelics and that IQ points probably wonât change. I think I misread you.
The larger part of my scepticism is my intuitive hunch that loads of people wont suddenly start taking psychedelics if theyâre legal/âdecrimed. This isnât a strongly informed judgement and I could probably change my mind if presented with compelling reasons.
On the worldview stuff, if the idea is something like âpeople take drugs and this changes how they think for the betterâ, which I actually think is pretty plausible, a particular challenge is that those who you, I expect, would most like to take such drugs, i.e. the very close-minded, are probably going to be the least likely to take them anyway.
Iâm not sure how the beliefs in Table 3 would lead to positive social change. Mostly just seems like an increase in some vague theism, along with
acceptance/âcomplacency/âindifference/ânihilism. The former is epistemically shaky, and the latter doesnât seem like an engine for social change.
Modeling the risk of psychedelics as nonexistent seems like a very selective reading of Carbonaro 2016:
âEleven percent put self or others at risk of physical harm; factors increasing the likelihood of risk included estimated dose, duration and difficulty of the experience, and absence of physical comfort and social support. Of the respondents, 2.6% behaved in a physically aggressive or violent manner and 2.7% received medical help. Of those whose experience occurred >1 year before, 7.6% sought treatment for enduring psychological symptoms. Three cases appeared associated with onset of enduring psychotic symptoms and three cases with attempted suicide.â
Thanks for doing this. I found the way youâd quantified the treatment benefits very useful.
Two sets of comments:
1
I would have found it very helpful if youâd explained the context of the intervention. Some questions that sprung to mind:
what are ballot initiatives in Calfornia and how do they work? is this to raise funds to put this up for a vote? If so, what are the costs involved and why do they vary? Is this to put this to put it up for a vote and campaign for it? Something else?
What exactly do you mean by âdrug liberalisationâ? Changing the medical laws, the recreational ones? If so, in what way? I understand you want to be âagnosticâ on the details, but Iâd say youâre closer to being âvagueâ as Iâm really not sure what you have in mind. Another way of being agnostic would be to say âthere are options A to F of how this could work in practice, we donât know which is best and it would/âwouldnât change the model for these reasonsâ
2
Iâd also have found it helpful if youâd explained what youâre doing at the various steps of the model and why. I assume that anyone, such as myself, interested to look through the model would also read a text explanation and this would aid comprehension. Important counterfactuals to consider seem to be:
-How many years of benefit there are before this happens anyway. Your answers were 2, 5 and 10 years. Could you explain your thinking there and what itâs sensitive to?
-Why did you model the effect just in California? I think most of the benefit of something like this is that it would speed up drug reform around the world, i.e. what California does today, the world does tomorrow. It would be good to have something, even vague, on how much better this looks if we include a domino effect.
-You state this would need $6m, $14m or $38m in funding. In addition to knowing exactly what that is funding for, Iâd want to know about the expected value at different sums raised. i.e. if I can just scrap together $100k, am I right in thinking this is a waste of time? If thatâs true, that changes who the relevant audience is and it would only be major funders.
-You mention depression but not, as far as I can see, anxiety. Is there a reason for that?
-Are you assuming the people who would be treated with psychedelics wouldnât otherwise receive treatment? What are you assuming here?
I might have other things, but thatâs probably enough for now.
In this model, what is the probability that the initiative (which I see is modeled as costing $6-39M) is successful? Or is it assumed that in the case where it isnât going to succeed, the cost is limited to the cost of polling ($50-300k)?
Did you collect base rate information for other initiatives before campaigns (which tend to lower approval relative to pre-campaign polling) for that parameter?
I echo Michael Plantâs sentiments. Iâm glad youâre quantifying the benefits of this potential intervention.
I started looking through the CEA and thought it seemed optimistic in various ways, but then I realized I could just look at the end number and see if, even without adjustments, it beat GiveWell recommended charities. Unfortunately it doesnât. You said that GiveWellâs charities are in the range of hundreds of dollars per DALY and that didnât gel with my memory. I looked it up and AMF is around $1,965 per life saved, equivalent to 36 DALYs, so 1,965â36 = ~$54/âDALY. SCI was $1,080 per life saved, so $1,080/â36 = $30/âDALY. GiveDirectly is $11,663/â36=~$323/âDALY, but the reason they recommend GiveDirectly is in part because it has a lower barrier to prove itself in evidence because it is a very âdirectâ intervention. (source: https://ââdocs.google.com/ââspreadsheets/ââd/ââ13b_qt-G_TQtoYNznNak3_5dzvzgCSUPJnk3l5dMisJo/ââedit#gid=1034883018) These numbers hold up roughly when you look at estimates from 2012 when they were still using DALYs.
Nevertheless, if you are considering where to donate, your best guess estimate is less cost-effective than GiveWell interventions. This is before it goes through the rigors of a GiveWell CEA as well, which would definitely have less optimistic numbers, especially given the low evidence base.
Iâd like to end on a note that I think that posting new cause areas to the EA movement is scary because itâs a critical minded bunch, so hats off to you for having the courage to do so. Keep trying; I commend you for it. Unfortunately, even if you defend it as well as it can be defended, it might not win compared to the existing top charities. However, if nobody does this work, no new causes will be âdiscoveredâ, and so even if it doesnât win, this sort of work is very likely to be net good in expectation.
The reason why interventions like AR or x-risk are accepted by the EA movement (although not by all EAs) is that from a CEA perspective they do better than GiveWell top charities. The reason a lot of people still donât accept them as interventions though is because people discount based on evidence base differently, with some people taking non-evidence based CEAs more seriously than others. If drug policy does worse from a CEA perspective than GiveWell, AR and x-risk, and is worse from an evidence perspective than GiveWell charities, where is its advantage?
You could make a case that itâs better from a metric perspective (ie preventing unhappiness through depression rather than DALYs which has issues with it, like over-valuing preventing death according to a lot of value systems), but deworming improves lives; it doesnât prevent death. Same with GiveDirectly.
For giving detailed feedback on the CEA, I unfortunately just donât have the energy to do the full thing, but if the final number still isnât enough to make me switch from GiveWell charities, it doesnât make sense to look more into the details. However, one thing that jumped out to me that others mentioned was the chance of the ballot coming through. I think looking up the historical rate of ballot initiatives being passed would be a good thing to look into.
On your original comparison to GW charities, I wouldnât just take GWâs analysis as the canonic truth on the matter. Their CEA is pretty complicated, but ultimately they value charities based on how well they either 1. save lives or 2. increase consumption.
What you think about the value of saving lives is a philosophical question. Iâve written about this elsewhere so wonât repeat myself. What you think about the value of increase consumption (SCI and GD) is probably an empirical question. If you value happiness then increasing consumption is a really bad way to increase aggregate happiness because of adaption and comparison effects (I discuss this in my EAG talk).
When I think about GW charities, Iâm am highly sceptical they do much good at all. I know, highly controversial....
I say this because it opens the space to look at other things, like mental health and pain, both of which drug policy reform help.
However, even if you take GWâs calcs at face value (Iâm not sure exactly what that is) I think it would still be possible to build an EV calc for drug policy reform that rivals them. In this post I suggest a campaign for rescheduling psychedelics could spend ÂŁ250 billion and be competitive with AMF. Milanâs calcs are really helpful because itâs important to start filling in the details of this analysis. In many ways, Milanâs is more complete than mine, which is quite simple.
Fair point, that deworming and cash transfers increase consumption instead of directly increase well being, or at least thatâs what GiveWellâs main analysis rests on. I do recall that the GD study actually did look at SWB and on page 4 (bit.ly/ââ2B97A1Y) it says that it increased a bunch of different happiness metrics as well (depression, stress, happiness and life satisfaction). However, if you only looked at that effect, GiveDirectly may not be that cost-effective. I havenât investigated it that much from that angle.
In terms of preventing infant mortality, it seems unlikely that losing a child wouldnât cause immense suffering to the parents, especially the mother. People often think that this wouldnât happen because people just âget used toâ babies dying, but the odds that a child will die is actually quite low nowadays, even in the developing world. In India, where I have the most experience, itâs measured in deaths per 1,000 live births, not 100, because itâs thatâs rare. Additionally, because I donât think death is nearly as bad as DALYs would have it, I looked a lot into parental mourning before choosing SMS reminders. I donât have anything formal I wrote up I can point to (though I might at some point), but my research found that most parents, after the loss of a child, are depressed for around a year, with some tail ends of people who never appear to recover.
If itâs the metrics issue thatâs leading to drug policy reform, I would recommend looking into preventing iron deficiency (through supplements or fortification) as an alternative. Itâs more evidence based and iron deficiency causes massive unhappiness. Anecdotally Iâve had friends who transformed from sad grumpy monsters into happy productive members of society after realizing they were deficient. Additionally thereâs evidence it increases income, increases IQ if taken during pregnancy, and decreases mortality in certain circumstances, so itâs pretty robust no matter the metrics you care about.
Lastly, Iâll admit that I havenât read all of your posts /â critiques of AMFâs effectiveness, so Iâll have to go and do that :)
If thatâs what you think does the real work of drug liberalisation, you should probably state that and build a cost-effectiveness model on that basis, rather than try to justify drug reform on other terms but with that as the true motivation. I, for one, am pretty sceptical, because I canât imagine loads more people would, counterfactually, start taking drugs recreationally and that, for those that do, this will have much impact on their cognitive powers.
Hmm. Yes, I agree cognitive shifts could be pretty powerful from psychedelics and that IQ points probably wonât change. I think I misread you.
The larger part of my scepticism is my intuitive hunch that loads of people wont suddenly start taking psychedelics if theyâre legal/âdecrimed. This isnât a strongly informed judgement and I could probably change my mind if presented with compelling reasons.
On the worldview stuff, if the idea is something like âpeople take drugs and this changes how they think for the betterâ, which I actually think is pretty plausible, a particular challenge is that those who you, I expect, would most like to take such drugs, i.e. the very close-minded, are probably going to be the least likely to take them anyway.
Iâm not sure how the beliefs in Table 3 would lead to positive social change. Mostly just seems like an increase in some vague theism, along with acceptance/âcomplacency/âindifference/ânihilism. The former is epistemically shaky, and the latter doesnât seem like an engine for social change.
Modeling the risk of psychedelics as nonexistent seems like a very selective reading of Carbonaro 2016:
âEleven percent put self or others at risk of physical harm; factors increasing the likelihood of risk included estimated dose, duration and difficulty of the experience, and absence of physical comfort and social support. Of the respondents, 2.6% behaved in a physically aggressive or violent manner and 2.7% received medical help. Of those whose experience occurred >1 year before, 7.6% sought treatment for enduring psychological symptoms. Three cases appeared associated with onset of enduring psychotic symptoms and three cases with attempted suicide.â
Thanks for doing this. I found the way youâd quantified the treatment benefits very useful.
Two sets of comments:
1
I would have found it very helpful if youâd explained the context of the intervention. Some questions that sprung to mind:
what are ballot initiatives in Calfornia and how do they work? is this to raise funds to put this up for a vote? If so, what are the costs involved and why do they vary? Is this to put this to put it up for a vote and campaign for it? Something else?
What exactly do you mean by âdrug liberalisationâ? Changing the medical laws, the recreational ones? If so, in what way? I understand you want to be âagnosticâ on the details, but Iâd say youâre closer to being âvagueâ as Iâm really not sure what you have in mind. Another way of being agnostic would be to say âthere are options A to F of how this could work in practice, we donât know which is best and it would/âwouldnât change the model for these reasonsâ
2
Iâd also have found it helpful if youâd explained what youâre doing at the various steps of the model and why. I assume that anyone, such as myself, interested to look through the model would also read a text explanation and this would aid comprehension. Important counterfactuals to consider seem to be:
-How many years of benefit there are before this happens anyway. Your answers were 2, 5 and 10 years. Could you explain your thinking there and what itâs sensitive to?
-Why did you model the effect just in California? I think most of the benefit of something like this is that it would speed up drug reform around the world, i.e. what California does today, the world does tomorrow. It would be good to have something, even vague, on how much better this looks if we include a domino effect.
-You state this would need $6m, $14m or $38m in funding. In addition to knowing exactly what that is funding for, Iâd want to know about the expected value at different sums raised. i.e. if I can just scrap together $100k, am I right in thinking this is a waste of time? If thatâs true, that changes who the relevant audience is and it would only be major funders.
-You mention depression but not, as far as I can see, anxiety. Is there a reason for that?
-Are you assuming the people who would be treated with psychedelics wouldnât otherwise receive treatment? What are you assuming here?
I might have other things, but thatâs probably enough for now.
In this model, what is the probability that the initiative (which I see is modeled as costing $6-39M) is successful? Or is it assumed that in the case where it isnât going to succeed, the cost is limited to the cost of polling ($50-300k)?
Did you collect base rate information for other initiatives before campaigns (which tend to lower approval relative to pre-campaign polling) for that parameter?